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Literature Review 

O ’Connor and Park (2001), echoed by 
Hemon and Schwartz (2003), have 
argued that the LIS field needs profession¬ 
als who can be both producers and critical 
consumers of research if it is to continue 
to evolve. Stephenson (1990) stated that 
library practitioners need to take a more 
active role in research and participate at a 
level that more accurately represents their 
numbers in the field. In 1990, at the time of 
Stephenson’s writing, a disproportionately 
small number of practitioners produced 
research while library educators and doc¬ 
toral students were greatly over-represent¬ 
ed. Research, Stephenson argued, “needs 
to be recognized as an integral part of a 
continuing process of exploration, analy¬ 
sis, planning, and growth that extends to 
all aspects and participants in the field.” 
(1990, p. 15). In the face of looming bud¬ 
get cuts, this is especially important be¬ 
cause collecting empirical data is one way 
that practitioners can demonstrate the ef¬ 
ficacy of their programs and help protect 
them against cuts (Feldman, 2010, 2011). 
In her survey of ALA-accredited MLIS 


programs across the US and Canada 
(1990), Stephenson found that only 14 of 
the 51 responding programs offered ad¬ 
vanced courses in research methods, de¬ 
spite recognition in the field that advanced 
courses are necessary if MLIS students are 
to become competent researchers. Twen¬ 
ty-four of the responding programs also 
did not require students in their basic re¬ 
search methods course to carry out any re¬ 
search. Of those that did include a research 
project, the survey was the most common 
method of data collection. Stephenson 
concluded that MLIS graduates were not 
consistently prepared with the skills or the 
inclination to make research a part of their 
professional lives. 

Park (2003) showed that many of Ste¬ 
phenson’s concerns were still valid over 
a decade later, and she recommended that 
ALA make research methods training nec¬ 
essary for program accreditation if MLIS 
programs continued to leave research 
methods out of the core curriculum. Park 
compared research methods courses across 
ALA-accredited MLIS programs and also 
across other professional disciplines at the 
same institutions (2003). In Stephenson’s 
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study, 35 (69%) of the responding pro¬ 
grams required research methods (1990); 
this number had decreased by the time of 
Park’s (2003) review, in which she found 
that 32 (59%) of the 54 programs exam¬ 
ined required research methods. The ac¬ 
creditation standards for business and 
social work programs, on the other hand, 
required research methods in all programs. 
Where there was a core research methods 
class in MLIS programs, Park found the 
content to be inconsistent and that qualita¬ 
tive methods were often given short shrift 
in favor of more traditional quantitative 
methods. Even though Powell (1999) not¬ 
ed the growing popularity of qualitative 
methods and multi-disciplinary research 
efforts in LIS research, Park’s (2003) re¬ 
search showed that these changes in the 
field did not appear to be reflected in the 
training of future LIS professionals. 

In 2010, the status of research methods 
in MLIS curricula remained relatively un¬ 
changed: 61% of ALA-accredited MLIS 
programs required research methods (Luo, 
2011). As part of the same study, Luo 
surveyed librarians about their involve¬ 
ment in research. The majority (84.3%) 
of Luo’s respondents were involved in or 
made use of research at work, although 
more were consumers than producers. Of 
the respondents, 51.4% of 554 had taken 
research methods in their MLIS program. 
In the opinion of 72.5% who answered, 
research methods should be a required 
MLIS course. 

The LIS literature provides insight into 
the content of two research methods cours¬ 
es. Liebscher (1998) described a course 
that included both quantitative and quali¬ 
tative methodologies and gave students 
the opportunity to experience as well as 
to consume research. Liebscher’s students 
furthered their understanding of research 
methodologies gained through classroom 
learning by using both quantitative and 
qualitative data collection and analysis 
methods in an authentic research project. 
Liebscher particularly emphasized the 
need for authentic experience when learn¬ 


ing qualitative methods, as it is impossible 
to learn the practical skills of observation 
and interviewing from dummy data. 

In contrast, Dilevko (2000) described a 
course and teaching strategy designed to 
prepare graduate students to be compe¬ 
tent researchers while reducing “statistics 
anxiety,” a common phenomenon among 
graduate students encountering statistics 
for the first time. Through analyzing and 
critiquing the use and analysis of statis¬ 
tics in newspaper and Library Quarterly 
articles, Dilevko’s students gained confi¬ 
dence and understanding of statistics and 
associated research methods. However, 
Dilevko’s course did not include practical 
experience of conducting field research or 
exposure to qualitative methods. 

Students as Researchers 

The research experiment reported in this 
article strove to remedy in one MLIS pro¬ 
gram the following deficits documented 
by the investigations of research courses in 
LIS schools over the past two decades: (a) 
an inadequate number of research courses, 
particularly advanced research courses; (b) 
a focus on one particular methodology, 
the survey; and (c) few reports of courses 
in which the students actually participated 
in authentic research projects. The intent 
in reporting this is to encourage other LIS 
schools to make known such opportunities 
if they exist in their MLIS programs, and if 
they do not, to consider incorporating them. 

The focus of this article is one offering 
of the University of Washington Informa¬ 
tion School’s (iSchool) recently developed 
advanced research seminar generically ti¬ 
tled Research in Action. The seminar fol¬ 
lowed Liebscher’s (1998) model in that it 
aimed to expose students to quantitative 
and qualitative methodologies through a 
mix of classroom learning and authentic 
practice. This course extended Liebscher’s 
model by including online students in the 
experiential learning. Research in Action 
also included a program evaluation com¬ 
ponent and contributed to students’ pro- 
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fessional development by exposing them 
to models of working partnerships and in¬ 
terdisciplinary collaboration that they will 
carry with them into their professional 
lives. 

This new course offering was timely 
in light of a survey of students, the com¬ 
munity, and employers conducted by the 
iSchool during Spring 2011 ahead of a 
curriculum revision. The survey indicated 
that the most desired but at the time un¬ 
available course offering was evaluation 
of programs. The research project with 
which Research in Action is affiliated in¬ 
cludes an element of library program eval¬ 
uation, so this iteration of the new course 
also answered the need raised by gradu¬ 
ates of the program. 

Research in Action had two goals. The 
first goal, described above, related to stu¬ 
dents’ research-relevant learning oppor¬ 
tunities and outcomes. The second goal 
related to the outcomes of the large-scale 
research project in which the students par¬ 
ticipated: we needed to train a team of re¬ 
searchers to carry out data collection and 
coding for the project. We approached 
the seminar with the following questions 
in mind: How can the advanced research 
seminar be an appropriate way to meet 
both the needs of the students and the 
needs of the research project? And, what 
evaluation methods can be used for deter¬ 
mining whether this is a successful experi¬ 
ment in learning? 

Lave and Wenger’s (1991) theory of sit¬ 
uated learning through legitimate periph¬ 
eral participation in a community of prac¬ 
tice provides an appropriate framework 
with which to describe our pedagogical 
approach. Drawing on the apprenticeship 
model of learning, Lave and Wenger see 
learning as a social practice; the social and 
environmental contexts in which the learn¬ 
ing takes place influence the outcome. 
Novices begin on the periphery of a com¬ 
munity of practice and, through interac¬ 
tions with more experienced members of 
the community and engaging in legitimate 
participation in the practice of the commu¬ 


nity, they transition from peripheral to full 
participation as their skills and knowledge 
grow. This conceptualization of learning 
is directly relevant to the goals and imple¬ 
mentation of Research in Action, where we 
anticipated that our students would grow 
from novices to full-fledged researchers 
through authentic practice as a community 
in a faculty research project. 

The Research Seminar: Design 

Research in Action is worth 4 quarter 
credit hours, split evenly over two quar¬ 
ters. The first quarter (2 credits) focuses 
on reviewing research methods to place 
this project in context and showing stu¬ 
dents how to use the specific tools for data 
collection. During the second quarter the 
students collect, code, and in some cases 
analyze, data. They use a class discussion 
board and group meetings every two or 
three weeks to share experiences. Resi¬ 
dential students and instructors meet in a 
classroom equipped with video cameras 
and microphones. The online students 
join the residential students via Adobe 
Connect, a videoconferencing platform 
that enables online students to see, hear, 
and participate in the classroom activi¬ 
ties in synchronous sessions. The focus is 
on highly interactive learning in a small- 
group setting to ensure students gain suf¬ 
ficiently deep understanding of the course 
content. 

The Research Project 

In this iteration of Research in Action, 
the community of practice was made up 
of researchers and partners involved in 
Project VIEWS 2: Valuable Initiatives in 
Early Learning that Work Successfully, an 
Institute for Museum and Library Servic¬ 
es (IMLS) National Leadership Research 
Grant initiative that began in October 
2011 and continues through September 
2014. VIEWS 2 followed and is based 
on Project VIEWS, an in-depth needs as¬ 
sessment of the early literacy landscape in 
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Washington’s public libraries funded by 
an IMLS National Leadership Planning 
Grant. In Project VIEWS, representatives 
from Washington’s public libraries, the 
Department of Public Instruction, the De¬ 
partment of Early Literacy, and numerous 
community partnerships focused on early 
literacy, participated in a 2-day planning 
meeting to conduct the needs assessment. 
The purpose of the VIEWS 2 research is 
to find valid, reliable, developmentally 
appropriate means for public libraries to 
measure the early literacy skills outcomes 
for children attending programs from birth 
to Kindergarten (K) and to strengthen and 
leverage public library and school partner¬ 
ships to improve early literacy practices. 

Several community partners are closely 
involved in VIEWS 2. Official partners are 
the Early Learning Public Library Partner¬ 
ship (ELPLP), consisting of 26 urban, sub¬ 
urban, and rural Washington public library 
systems; the Washington State Library; 
and the Washington Foundation for Early 
Learning, a non-profit organization. Other 
national and local partners are represent¬ 
ed on a Strategic Advisory Board (SAB). 
While all partners share the role of provid¬ 
ing insight and feedback on grant matters, 
each partner has also taken on additional 
roles in the management of the grant. The 
University of Washington Information 
School, as the lead partner, oversees the 
overall and day-to-day management and 
implementation of the grant and provides 
the course described to train student re¬ 
searchers. The ELPLP is providing the li¬ 
brary sites and, along with the Washington 
Foundation for Early Learning, assists by 
communicating with and updating these 
sites. The Washington State Library, due 
to the program officers’ experience with 
training public librarians around the state, 
is assisting with the implementation of the 
intervention in the grant. Finally the SAB, 
which includes individuals from these 
partners as well as many others, serves 
as the formal organization for providing 
formative advice and insight on any grant 
matters. 


The first offering of the advanced re¬ 
search seminar took place in 2012 for pre¬ 
intervention data collection for VIEWS 2. 
An intervention, based on the results of the 
first data collection, will follow this first 
offering and the course will be offered 
again in 2013 for post-intervention data 
collection. Some of the same students will 
participate in data collection while others 
will join the project in progress. 

The Research Seminar: 
Implementation 

In this version of the advanced research 
seminar, students learned about pre- and 
post- experimental design and sampling in 
the context of the VIEWS 2 project. This 
type of research design is rare in MLIS 
programs and in published LIS research. 
The VIEWS 2 project uses both qualita¬ 
tive and quantitative methods, giving stu¬ 
dents a taste of mixed methods research. 

The first quarter began with a discus¬ 
sion of the distinction between academic 
research and evaluation. This is important 
to students who, as professionals, will 
likely need to become proficient in under¬ 
standing and conducting both types of in¬ 
vestigations (Luo, 2011). Project VIEWS 
2 is particularly well suited for this discus¬ 
sion as it features program evaluation em¬ 
bedded in academic research. 

One course assignment included com¬ 
pleting the university’s online human sub¬ 
jects training, required of all researchers. 
Understanding the ethical implications of 
research is particularly important in the 
context of VIEWS 2, where the research 
is conducted with young children, a vul¬ 
nerable population, and the graduate stu¬ 
dents collect data with no on-site supervi¬ 
sion. Human subjects training also made 
students more sensitive to, and better able 
to handle, questions and concerns raised 
by librarians or parents at the story times 
they observed, and it increased awareness 
of ethical issues students might encounter 
in the future when conducting or reading 
research as practitioners. 
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In line with the focus on formative as¬ 
sessment and feedback, students’ exposure 
to the instruments in the first quarter that 
they used in the second quarter, and the 
development of their observational skills, 
was iterative. One of the instruments, the 
Benchmarks Curricular Planning and As¬ 
sessment Framework (BCPAF), was de¬ 
veloped and implemented prior to the start 
of VIEWS 2 (Feldman, 2010; 2011) and 
is based on the Washington Department of 
Early Learning guidelines for early learn¬ 
ing. It is used to record observations of 
child behavior, aggregated at the group 
level. A second instrument, the Program 
Evaluation Tool (PET), was developed by 
the VIEWS 2 research team and was de¬ 
veloped as a complementary instrument to 
BCPAF with a similar structure and paral¬ 
lel, as far as possible, content; it is used 
to record observations of program delivery 
and content. 

Early in the first quarter students ob¬ 
served a library storytime of their choice 
to give them experience of some of the 
practical issues of observation, such as 
where to situate themselves and how to 
take field notes. Later in the quarter, once 
they were familiar with the specific instru¬ 
ments, they participated in several cycles 
of practicing using the instruments with 
videos of library storytimes followed by 
feedback on how to improve their skills. 
This ensured that the students were well- 
practiced, competent observers before go¬ 
ing out into the field to collect data for the 
project. 

The first quarter of the seminar cov¬ 
ered a great deal of content in a fairly 
short time. To deal with this challenge, 
the structure of the class placed the re¬ 
sponsibility for acquiring “the facts,” or 
the content knowledge, on work by the 
students outside of class. Class meetings 
were supplemented by carefully selected 
readings, brief, pre-recorded lectures, and 
coding videos of storytimes, for which stu¬ 
dents filled out reflection sheets regarding 
any questions they had about where their 
codes differed from those on the answer 


keys they were given. This left class meet¬ 
ing times entirely free for discussion and 
collaborative learning. Students were en¬ 
couraged to share their questions and con¬ 
fusions about the content knowledge dur¬ 
ing class times so that they could all learn 
from each other’s experiences. In addition, 
each student was assigned a mentor from 
the instruction team who was on hand to 
provide individual help and support as 
needed. Mentors were tasked with collat¬ 
ing commonly occurring issues or con¬ 
cerns among their mentees, which were 
then used to structure class discussion. 

This process of coding, reflecting, and 
discussing led to a number of changes to 
the VIEWS 2 coding manual ahead of data 
collection in the spring. Lave and Wenger 
(1991) caution that existing power struc¬ 
tures within communities of practice, such 
as the traditional relationship between in¬ 
structor and student, can inhibit learning- 
through-doing if not properly managed. 
Encouraging students to question the cod¬ 
ing manual and offer improvements to it 
increased their confidence as members of 
an authentic research team. The questions 
and issues that students raised showed a 
deep understanding of what the VIEWS 
2 project seeks to measure and also ulti¬ 
mately improved the research tools. 

In the first quarter, emphasis was placed 
on becoming reliable using the research 
instruments. Inter-coder reliability is an 
issue of great importance in real-world 
research such as VIEWS 2 but is difficult 
to achieve using dummy data or isolated 
individual research projects where reli¬ 
ability is less of an issue. Experiencing the 
process of becoming reliable on qualita¬ 
tive (i.e., taking field notes and preparing 
interpretative descriptions from them) and 
quantitative (i.e., coding observed behav¬ 
iors against established indicators) tools 
that they subsequently used prepared stu¬ 
dents to conduct qualitative and quantita¬ 
tive research in future professional or aca¬ 
demic capacities. 

In the second quarter, students applied 
what they learned in the first quarter as 
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they collected and coded data in the field. 
Liebscher (1998) defines the essentials 
of education in qualitative methods as 
“[1] defining (and justifying) purposive 
samples, [2] data collection through inter¬ 
views and observation, [3] data analysis 
simultaneous with data collections, and 
[4] data analysis through reduction and in¬ 
terpretation.” The first essential, sampling, 
is covered in the first quarter as part of the 
background of VIEWS 2. The remaining 
essentials, data collection, simultaneous 
analysis, and interpretation, were the em¬ 
phases of the second quarter. 

The students independently collected 
and coded data in the field. Focusing on 
a stratified random selection of 40 public 
libraries throughout the state of Wash¬ 
ington, the students carried out field ob¬ 
servations of three storytimes per library, 
taking field notes and interpreting their 
data. They coded their data using the 
BCPAF and PET instruments developed 
and validated by the research team. Online 
students received exactly the same train¬ 
ing in research methods as the residential 
students. Equipment was sent to them via 
UPS. The randomly-chosen libraries are 
scattered throughout Washington, so on¬ 
line students were particularly useful to 
collect data in places many miles from the 
university. 

Each observed storytime during the sec¬ 
ond quarter was videotaped with two cam¬ 
eras, one focused on the children and one 
on the delivery of the storytime. The cod¬ 
ing took place both directly after the story¬ 
time for BCPAF and from the videos for 
both BCPAF and PET. This also enabled 
online students outside the state to partici¬ 
pate in the research experience. Although 
some of the most remote students were 
unable to visit storytimes themselves, for 
example, a student in Alaska and one in 
Washington, D.C., they were still able to 
code videos taken by other students. 

Partnership and inter-organization col¬ 
laboration make VIEWS 2 possible and 
these themes ran through both quarters 
of Research in Action. In the first quar¬ 


ter, students learned that partnership with 
the ELPLP and the Foundation for Early 
Learning contributed to the development 
of the grant by acting as a liaison with their 
member libraries and providing resources 
as well as valuable insight and ideas during 
the planning of the grant. The students saw 
that the partnership is ongoing throughout 
the grant, as the ELPLP and the Founda¬ 
tion for Early Learning have committed 
themselves to the success of the grant, 
encouraging and achieving full participa¬ 
tion from all member libraries. In the sec¬ 
ond quarter, students saw some aspects of 
these partnerships in action as they visited 
libraries across Washington State and in¬ 
teracted directly with librarians. 

The course instructors came from a 
range of backgrounds and included a fac¬ 
ulty member and doctoral students from 
the iSchool and a postdoctoral researcher 
from the College of Education. The course 
utilized the interdisciplinary strengths of 
the different instructors. For example, to 
be successful as observational researchers 
in the context of early literacy programs 
in public libraries, the seminar students 
not only needed to know how to apply 
observation as a research method, they 
also needed an understanding of informal 
learning in public libraries, child devel¬ 
opment, and early literacy more broadly. 
They needed to be able to recognize de¬ 
velopmental and literacy milestones when 
observing the diverse children spanning 
a range of ages that attend library story 
times. The postdoctoral researcher who 
participated in all class sessions has a mas¬ 
ter’s degree in developmental psychology. 
As a specialist in early learning, she was 
well placed to help the seminar students 
gain the background knowledge in child 
development and early learning that made 
them better observational researchers in 
the context of VIEWS 2. As the developer 
of the Benchmarks Curriculum Planning 
and Assessment Framework (Feldman, 
2010; 2011), she was deeply involved with 
their reliability training. 

The course is offered as credit/fail. As 
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the emphasis is on authentic research ex¬ 
perience, or legitimate participation to use 
Lave and Wenger’s terminology, students 
were assessed in the same way that non¬ 
student researchers working on the proj¬ 
ect would be; each achieved reliability 
through a valid testing procedure. Grad¬ 
ing the course credit/fail also allowed stu¬ 
dents to concentrate on becoming skilled 
researchers rather than maintaining their 
grade point average. The close coupling of 
students’ learning goals and the progress 
of the VIEWS 2 research project was a pri¬ 
mary motivation for restricting the course 
to a small, interactive seminar rather than 
a lecture as it is vital that the instructors 
had the time and resources to be attentive 
to the learning needs of each individual 
student. 

Results 

The outcomes brought a positive re¬ 
sponse to both of the research questions: 
How can the advanced research semi¬ 
nar be an appropriate way to meet both 
the needs of the students and the needs of 
the research project? And, what evalua¬ 
tion methods can be used for determining 
whether this is a successful experiment in 
learning? The seminar was successful in 
meeting both the needs of the students as 
apprentice researchers and the needs of 
VIEWS2. The description of the design 
and implementation of the course answers 
the first question. The success for the re¬ 
search project was demonstrated by the 
outcome that the second quarter data col¬ 
lection and coding proceeded smoothly 
and in accordance with the requirements 
of the grant. The success for students was 
measured by the inter-coder reliability es¬ 
tablished with Pearson R, % agreement, 
and Kappa applied to their codings of 3 
videos. The other means of assessing out¬ 
comes was through their evaluations of 
the course. The numerical evaluation was 
solidly in the ‘very good’ category. The 
enthusiasm of students for this type of au¬ 
thentic research experience is found in the 


following comments drawn from the stu¬ 
dents’ anonymous responses on the open- 
ended portion of the course evaluations: 

• “1 am so excited about this course, it is 
a great realistic learning opportunity.” 

• “1 love this chance to interact with kids 
and observe the impact of early literacy 
programming in public library settings.” 

• “1 am very interested in this opportu¬ 
nity to assist with your research. It is 
extremely helpful for me to be involved 
in a project outside the College of Ed.” 

• “This class has been wonderful, 1 am 
thrilled I can take it!” 

• “1 took the basic research methods 
class, but this opportunity makes re¬ 
search so much more realistic and easy 
to understand.” 

• “This class taught me a lot about what 
goes into research and all the different 
things to take into account when doing 
it. 1 also learned about the variety of 
storytimes present and the different 
early literacy skills that children have.” 

Discussion 

Some of the details of the research sem¬ 
inar are, of course, specific to the VIEWS 
2 context. However, other LIS schools 
seeking to offer a similar course can draw 
on several aspects of this offering, confi¬ 
dent that it can be a successful experience 
and that it is theoretically sound pedagogy. 
The following recommendations may be 
helpful in planning such a research meth¬ 
ods course: 

• Coupling classroom-based learning 
with the opportunity to work on authen¬ 
tic faculty research projects provides 
valuable experiential learning that 
cannot be gained using dummy data or 
learning entirely confined to the class¬ 
room or desk. This pedagogical model 
is supported by Lave and Wenger’s 
(1991) theory of learning through le¬ 
gitimate peripheral participation. 

• Research projects that utilize mixed 
methods are particularly well suited 
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to learning experiences as students get 
to know how various methods work 
together in context. 

• The addition of a number of students 
to the research team helps further the 
research effort. For VIEWS 2, online 
students in remote parts of the state and 
across state lines were invaluable in 
reaching study sites far from a univer¬ 
sity’s campus. Input from the students 
also improved the project’s research 
tools. 

• Including researchers from relevant 
outside disciplines in the instruction 
team supports broadening students’ 
learning experience. Exposing students 
to interdisciplinary and/or collaborative 
research provides them with a model 
that will continue to be relevant as they 
become professionals in a field that is 
essentially interdisciplinary. 

• For online or distance programs, 
remote students can gain access to 
opportunities to work on authentic 
research with support from faculty. 
Online students are at risk of being left 
out of experiential learning due to lo¬ 
gistical challenges. Current technology, 
however, affords many ways to include 
geographically distributed students as 
full members of a research and learn¬ 
ing community. Our experience using 
videoconferencing software showed us 
that it is important to allow time for all 
participants to become familiar with it 
and to establish a protocol for smooth 
communication in a mixed online and 
offline environment. 

Conclusion 

Research in Action, the advanced re¬ 
search seminar described in this article, 
is an experience that provides students 
with both theory and genuine practice in 
research methods. It is a practice that can 
be modified and implemented in any LIS 
curriculum in which a faculty member is 
involved in data collection. The students 
enrolled in the course had differing aca¬ 


demic and professional goals but all stu¬ 
dents were able to benefit from the experi¬ 
ential learning offered by this seminar. For 
the students intending to become librar¬ 
ians, the training they received in program 
evaluation for early literacy activities in 
public libraries will enable them to better 
serve their libraries’ youngest users. The 
exposure to different evaluative tools for 
use in informal learning environments in¬ 
creased their awareness and use of such 
tools. For the students intending to contin¬ 
ue their academic studies, the hands-on re¬ 
search experience will help to make them 
more competitive applicants to doctoral 
programs. The seminar also stands to ben¬ 
efit project partners, as their organizations 
will gain from the research and evaluation 
outcomes. 

Training in research methods is an im¬ 
portant part of LIS graduate education. 
Prior work by Stephenson (1990), Park 
(2003), and Luo (2011) has shown that re¬ 
search methods in LIS programs is often 
missing or, when present, fails to represent 
the range of methods employed in library 
and information research today. A combi¬ 
nation of classroom learning and authentic 
participatory experience with a collabora¬ 
tive research project prepares students to 
be both consumers and producers of re¬ 
search, familiar with multiple methodolo¬ 
gies. 
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